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ABSTRACT 

Virtual human simulation integrated into virtual reality ap- 
plications is mainly used for virtual representation of the user in 
virtual environment or for interactions between the user and the 
virtual avatar for cognitive tasks. In this paper, in order to pre- 
vent musculoskeletal disorders, the integration of virtual human 
simulation and VR application is presented to facilitate physi- 
cal ergonomic evaluation, especially for physical fatigue eval- 
uation of a given population. Immersive working environments 
are created to avoid expensive physical mock-up in conventional 
evaluation methods. Peripheral motion capture systems are used 
to capture natural movements and then to simulate the physical 
operations in virtual human simulation. Physical aspects of hu- 
man 's movement are then analyzed to determine the effort level 
of each key joint using inverse kinematics. The physical fatigue 
level of each joint is further analyzed by integrating a fatigue 
and recovery model on the basis of physical task parameters. All 
the process has been realized based on VRHIT platform and a 
case study is presented to demonstrate the function of the phys- 
ical fatigue for a given population and its usefulness for worker 
selection. 



1 Introduction 

Although automation techniques have been used more and 
more prevalent in industrialized countries, there are still large 
requirements of manual handling operations, especially for as- 



sembly and maintenance operations. For this reason, ergonomic 
oriented design should be carried out to take account of human 
capacities and limitations as early as possible. Therefore, work 
design and evaluation is very important to avoid potential muscu- 
loskeletal disorders (MSDs) and to determine appropriate work- 
rest schedule. 

In order to accelerate the ergonomic evaluation procedure, 
digital human modeling (DHM) techniques have been developed 
and integrated into computer aided design systems. In this way, 
work evaluation can be carried out for the overall population us- 
ing anthropometric database; the high cost of physical mock- 
up can be avoided; the evaluation efficiency is much more im- 
proved in comparison to conventional evaluation approaches in 
workplace; potential ergonomic issues can be decreased. There 
are several commercial tools available in the market, for exam- 
ple, 3DSSPP |TTJ, Jack 0, AnyBody 0, Santos™ 0, etc. In 
these DHM tools, a simulated human is often used as a geo- 
metric representation, visual appearance of the real human be- 
ing in computer graphical simulation [5 1, and furthermore some 
ergonomic and biomechanical analysis can be carried out using 
those tools |6][7). However, as pointed out by |8|, this approach 
cannot provide a mimic motion due to its robotic background. 
Sometimes, it requires to place the manikin in the simulation en- 
vironment and to define all the motions manually, which is time 
consuming as well. 

With the development of computer graphic techniques and 
human computer interaction techniques, virtual reality (VR) 
techniques have been used widely in different application fields, 
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such as way finding and navigation techniques, object selection 
and manipulation, education, training, etc. Integrating visual, au- 
ditory and haptic systems, they are used to enable interactions be- 
tween virtual world and the user ll9l [l0l[TTl . Since 1990s, VR has 
been used in product design and manufacturing to facilitate the 
manufacturing procedure simulation, from the very initial con- 
ceptual design via operations management to final manufactur- 
ing processes lfl2l[T3l . Meanwhile, VR technique can also be 
used crossing different levels of an enterprise, form physical op- 
erations (e.g. virtual assembly M141I15I ) to human resource plan- 
ning [16|. Due to its immersive simulation and advanced inter- 
action peripherals, it is believed that VR system can be used for 
ergonomic analysis 11171 . However, there are still some limita- 
tions for a VR system. Firstly, to create an immersive virtual en- 
vironment with a high level of presence is very expensive. Also, 
it is often very difficult to obtain force interactions between the 
user and the virtual environment (VE) with high fidelity. Sec- 
ondly, even using a VR system with high level of presence, it is 
still not easy to carry out ergonomic analysis for a given popula- 
tion, since different subjects might have different anthropometric 
parameters and different biomechanical parameters. With the en- 
gagement of only several subjects in VR applications, it cannot 
reveal all the design flaws. 

Integrating virtual human techniques into VR applications 
can benefit the advantages from both sides fl8l . Several studies 
have been done in the field of ergonomics [19|, biomechanical 
analysis l20l . and other aspects ETl . Till now, most of the trials 
focus on the visual representation of virtual human in VR 11221 . 
where virtual human is used to increase the presence level of the 
VR system. In our study, we are trying to use this approach for 
physical fatigue analysis of static manual handling operations, 
since it is believed that physical fatigue might be a very impor- 
tant reason resulting in MSDs. In this paper, the detailed re- 
search effort in combining virtual reality techniques and digital 
human simulation tools is described. In Section |2j a virtual real- 
ity platform for all the research activities is introduced, including 
its hardware structure and its peripheral systems; Section [3] de- 
scribes briefly the modeling procedure of a virtual human and its 
work flow for fatigue analysis; Section@]refers to an application 
case in EADS. 



2 Virtual Reality Platform 

The VR platform is established in the laboratory of Virtual 
Reality & Human Interface Technology (VRHIT) in Tsinghua 
University. The platform is developed for doing research on work 
design in virtual working environment. The overall system struc- 
ture is illustrated in Figure Q] 

The virtual simulation system is equipped with simulation 
computers, a wide screen projection, and 5DT HMD 800-26. 
MultiGen Creator is used to create virtual objects, while Vega 
is to employ MultiGen Creator to convert a CAD model into a 




FIGURE 1 . Illustration of the hardware system for the VRHIT plat- 
form 



virtual reality model [23|. The virtual simulation system is able 
to load VR models and display via either the projection system or 
HMD. Stereo display is possible for some desktop applications. 

Different peripherals attached to this system include mo- 
tion tracking systems and data gloves. Two magnetic tracking 
systems (Fastrak® and Potroit® of Polhemus co.) and a self- 
developed optical motion tracking system are available for this 
platform. A participant wearing the motion capture suit and per- 
forming certain physical tasks is shown in Figure |2] The optical 
capture system is equipped with eight CCD cameras around the 
work space. In our case, some operations with high physical 
demands are under consideration, and in these operations, only 
static operations or slow movements are involved or considered. 
The overall capture system works at the frequency 25 Hz, which 
satisfies the minimum requirement to provide sufficient update 
rate of the simulation image, especially for quasi-static postures, 
and it provides sufficient detailed analysis of the static human 
body motion. Two data gloves, a 5DT Data Glove 5 Ultra and a 
self-developed data glove with haptic feedback, are in our pos- 
session to provide detailed gesture of the hands and feedback 
information while manipulating virtual objects. 

The objective of the VRHIT experiment platform is to en- 
able interactions between human and the simulated manufactur- 
ing environment and to realize interactive industrial work design 
taking consideration of human factors and ergonomics. This plat- 
form has been used in ll24l . ll25ll . and 112611 . for verifying a control 
panel design, assembly work design in heavy industry, and work 
efficiency evaluation, respectively. In this paper, it is mainly used 
to capture the static posture of a simulated drilling operation (see 
Figure [3(a)| . The posture is then analyzed by inverse dynamics 
based on digital human modeling techniques. 
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FIGURE 2. Self-developped motion capture system and its simula- 
tion system 




(a) subject (b) Virtual environment 



FIGURE 3. Subject wearing HMD in virtual environment 

3 Digital human modeling 

In this study, the human body is kinematically composed 
of a series of revolute joints. The Modified Denavit-Hartenberg 
(modified DH) notation system 1 27 1 is used to describe the move- 
ment flexibility of each joint and the linkage relation among the 
joints. According to the joint function, one natural joint can be 
decomposed into one, two, or three revolute joints. Each revolute 
joint has its rotational joint coordinate, labeled as qi, with joint 
limits: the upper limit qf and the lower limit qf . A general co- 
ordinate q = [qi,qj, ■ ■ ■ ,q n ] is defined to represent the kinematic 
chain of the skeleton. Overall, a virtual human is geometrically 
modeled by n = 28 (3 in shoulder, 2 in elbow, 3 in thigh, 1 in 
knee, 1 in ankle, 2 in waist, 4 in spine, 2 in neck) revolute joints 
to reproduce the mobility of all the key joints on the human body. 
The skeleton system (in Figure [U of the virtual human is graph- 
ically modeled using OpenGL and controlled by direct kinemat- 
ics, inverse kinematics, or loading motion capture data. 

The dimension and dynamic parameters of each body part 
can be obtained or calculated from anthropometry database, and 
meanwhile biomechanical strengths of each joint can be obtained 
from the literature in Occupational Biomechanics (TJ. In this 
way, the virtual human is modeled as a multi-rigid-body system, 
and the forces and torques at each key joint can be determined 
using inverse dynamics. 

The work flow for physical fatigue analysis of manual han- 
dling tasks is shown in Figure [5] In order to carry out work 




FIGURE 4. Geometrical modeling and graphical modeling of the vir- 
tual human 



analysis, it is necessary to collect human motion and interaction 
between human and the environment. In our case, it is possi- 
ble to obtain human motion from motion capture system, and 
then the motion information is transferred in the form of gen- 
eral coordinates q. Human motion can also be obtained via hu- 
man simulation tools. Large modeling efforts have been done in 
IOWA Santos and HUMOSIM EH to predict realistic postures 
and motions. Based on the techniques used in these two projects, 
a multi-objective posture prediction method is used to predict re- 
alistic postures with consideration of fatigue in ||29l . Interaction 
information is obtained either via haptic interface or via com- 
puter simulation. In some cases, especially for cases with high 
physical demand, the VR system is not able to provide precise 
and accurate force feedback to the user, and the external load is 
estimated or provided by real objects. In this way, inverse dy- 
namics can be carried out to determine the force and the torque 
at each joint. After determining the force and the torque load at 
each joint, it is possible to carry out joint fatigue analysis. If it 
goes further, it might be possible to determine the force of each 
engaged individual muscle as well. 

Physical fatigue is defined as "any exercise-induced reduc- 
tion in the capacity to generate force or power output" [ 30 1 . After 
knowing the physical load at each joint and the joint's strength, 
the muscle fatigue model proposed in [3T| is used to assess the 
physical fatigue during manual handling operations. This model 
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strength under different conditions in the literature. However, 
these models are not sufficient enough to construct a complete 
database for a given population, since there are lots of influencing 
factors determining the joint strength. Thus, more effort should 
be contributed to gathering all necessary data. Concerning the 
fatigue resistance and recovery rate, experimental validation is 
still under running to complete the validation of fatigue/recovery 
model. 

For each joint, after a manual handling operation, the fa- 
tigue could be assessed using a fatigue index which indicates the 
change of the maximum joint moment strength (Eq. [3j. 



FIGURE 5. Work flow for biomechanical analysis 



fatigue = 



r — r 

A max L ce. 



(3) 



can be described by a differential equation in Eq[T] 



decern \$ ) . ^cem \f) ^ , * 

~T f = ~ fc ~F 1 loadV) 

dl 1 max 



(l) 



Where T max represents the maximum joint moment capacity; 
V cem represents the current maximum joint moment capacity dur- 
ing a static exertion; Ti oa d is static joint moment load; k repre- 
sents the joint specified fatigue rate and it could vary across in- 
dividuals and joints, m = l/k indicates the fatigue resistance of 
a given joint. Using this model, the external task related parame- 
ters can be taken into consideration to assess the fatigue of a joint 
during a given manual handling operation. 

Meanwhile, a muscle recovery model using the same param- 
eters is used to describe the recovery process of the joint strength 
during rest period. It is mathematically described using Eq. |2j 
where R is the recovery rate. This model is extended from a re- 
covery model in ll32ll . 



decern (/ ) 

dt 



(2) 



It should be mentioned that k and R are parameters to de- 
scribe the fatigue properties of an individual or a population. 
They have almost the same function as the height and weight. 
However, they are very difficult to be measured directly. For 
the fatigue resistance, it has been mathematically analyzed in 
||3Tl[33l . For shoulder joint, it could be regressed from several 
existing maximum endurance time models (MET), and it has a 
theoretical value 0.7562 (SD=0.4347). 

In order to carry out the fatigue evaluation around a joint, it 
is necessary to obtain the necessary strength data and endurance 
data for a population. Currently, there are several biomechan- 
ical models available after the measurement of maximum joint 



4 Application 

4.1 Drilling case in EADS 

A drilling case in European Aeronautic Defence and Space 
(EADS) is taken for the case study. The drilling operation is 
graphically shown in Figure[6] The task is to assemble two fuse- 
lage sections with rivets. One part of the job consists of drilling 
holes all around the aircraft cross section. The number of the 
holes could be up to 2,000 on an orbital fuselage junction of an 
airplane. The drilling machine has a weight around 5 kg. The 
drilling force applied to the drilling machine is around 49 N. In 
general, it takes 30 seconds to drill a hole. There are several 
ergonomic issues due to the high physical demands and vibra- 
tion in drilling operations, and the physical fatigue caused by the 
physical operation might be one limitation for the work schedule 
design. 

This drilling task is simplified to two study case. One case 
is to assess the fatigue of a continuous drilling task with duration 
180 seconds. The other case is to select suitable workers for a 
drilling task with five work cycles (60 second drilling and 30 
seconds rest). 

4.2 Case 1 : fatigue assessment 

The first case at first carried out using the virtual reality plat- 
form. A total of 40 male industrial workers participated in the 
experiment. They were asked to maintain a simulated drilling 
operation for 180 seconds. During this period, the subjects were 
asked to keep the static posture as well as possible, while the 
posture was captured by magnetic tracking devices and mapped 
to virtual human for further theoretical fatigue analysis. 

During the drilling operation, the load of the drilling ma- 
chine was provided by a simulated drilling tool, which was made 
of concrete with a weight around 2.5 kg to avoid magnetic dis- 
tortion caused by ferrous material. Maximum output strengths 
were measured by a force measurement device before and right 



4 



Copyright ©2010byASME 




FIGURE 6. Drilling operations in the case study 



after the simulated drilling operations. Physical fatigue was char- 
acterized by the reduction of the joint strength along time rela- 
tive to the initial maximum joint strength (see Eq. [3). It has 
been found that three out of the 40 subjects could not sustain 
the external load for 180 seconds, and 34 subjects had a shoul- 
der joint fatigue resistance (Mean=1.32, SD=0.62). The physi- 
cal strength has been measured in simulated job static strengths, 
and the reduction in the operation varies from 32.0% to 71.1% 
(Mean=53.7% and SD=9.1%) l33l . 

Meanwhile, the same drilling operation was also analyzed 
in virtual human simulation. In order to simplify the simulation 
process, the average posture (flexion angles of elbow and shoul- 
der) among the 40 subjects was used to determine the shoulder 
joint strength according to Occupation Biomechanics |fl~|. The 
fatigue resistance for calculating the fatigue theoretically was 
obtained by comparing the theoretical fatigue model with other 
models in the literature (m=0.7562, (7=0.4347) (33]. It is be- 
lieved industrial workers have a higher fatigue resistance due to 
the training effect 041 . For this reason, the simulation was ex- 
ecuted with fatigue resistances superior to the average fatigue 
resistance, and the reduction of the shoulder joint strength was 
given in Table [T](Mean=5 1.7%, SD=12.1%). It has been found 
that the simulated result was similar to the realistic measured 
fatigue, which means the theoretical approach in virtual human 
simulation might give an acceptable assessment for the specified 
drilling operation of a given population. 

In the simulation of this case, only different shoulder joint 
strengths and different shoulder fatigue resistances were taken 
into consideration. Some other influencing factors were ne- 
glected, for example, different body weights, different heights, 
and some others. In fact, although there are definitely other fac- 
tors influencing the simulation result, joint strength and joint fa- 



TABLE 1. Normalized torque strength reduction for the population 
with higher fatigue resistance. S and o": mean and SD of shoulder joint 
moment strength; m and o m : mean and SD of fatigue resistance 
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tigue resistance are two most important parameters for fatigue 
analysis. The following subsection will show the effects of dif- 
ferent factors. 

4.3 Case 2: worker selection 

The second case was studied to verify the possible usability 
of virtual human simulation for worker selection. Worker selec- 
tion is often a practical problem during work design. In work 
design, the task has often been predefined, and there might be no 
possibility to modify the task parameters due to strict constraints. 
In this situation, designers have to choose suitable workers fit- 
ting the task. In ergonomics, generally, the anthropometric pa- 
rameters (e.g., height, weight), motion range, and biomechanical 
parameters (strength) are often used to create some criteria to se- 
lect the worker [ 1 j. For operations with high physical demands, 
four parameters (height, weight, strength, and fatigue resistance) 
might generate potential influences on the selection. 

A drilling case was simulated with the help of digital human 
simulation to demonstrate effects of those parameters on physical 
fatigue operations. The drilling operation is composed of five 
work cycles. In each cycle, the worker has to perform the drilling 
operations for 60 s continuously, and then takes a rest for 30 s. 
The total working time is 450 s. The selection criteria would be 
to find the workers who might be able to complete the given task. 

In the simulation, the variation among the population 
is taken into consideration, since different heights, weights, 
strengths, and fatigue rates might generate potential influences 
on the evaluation results. A virtual human was created using dif- 
ferent anthropometric data (height, weight) to cover 95% of the 
population, while the average value and the standard deviation 
of the strength and fatigue resistance were also taken into con- 
sideration. For each factor, there are three levels: average level, 
lower level, and higher level. For height, weight and strength, the 
lower level and the higher level were (x — 2a) and (1 + 2(7), re- 
spectively, (x — a) and (x+ a) were used to represent the lower 
level and the higher level of the shoulder fatigue resistance. Us- 
ing these factors at three different levels, the worker could be 
categorized into 8 1 different subgroups, and each subgroup has 
its own properties accordingly. 
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The maximum endurance time was predicted using the fa- 
tigue model and recovery model mentioned before, and it was de- 
termined while the joint capacity falls down to the current torque 
load. There are totally 8 1 endurance results for the drilling task, 
and the endurance results of the shoulder joint are shown in Table 

m 

Height and weight, both factors could generate slight influ- 
ences on the endurance time. It could be observed that smaller 
people have longer endurance time in comparison to taller ones 
with the same strength and the same fatigue resistance, since the 
moment arm might be shorter. The moment load can also be 
influenced by the weight. Heavier ones might generate larger 
moment around joint and then shorten the endurance time. How- 
ever, it should be noticed that the influence on the endurance 
time caused by the height and weight are relatively slight, and 
it can neglected. Therefore, these two factors are not significant 
parameters to choose suitable workers for a purely physical task. 

Obviously, the endurance is greatly influenced by the joint 
strength and the fatigue resistance. Larger joint capacity results 
in much longer endurance time at the same fatigue resistance 
level. Clearly, larger fatigue resistance enables us to maintain the 
physical task for a longer time at the same strength level. This 
observation conforms to our daily experience. Furthermore, if 
a worker could be characterized using the fatigue resistance and 
the joint strength, that means it might be promising to locate the 
worker in the 81 categories. In this way, suitable worker could 
be selected for a given physical task. 

The validation of the second case is still under construction, 
and the method and the difficulties will be discussed below. For 
the validation, the fatigue resistance, the recovery rate, and the 
joint moment strength of a subject should be measured. How- 
ever, it is difficult to determine the joint moment strength since it 
depends on several factors, such as posture, movement velocity, 
etc |[3"5ll . It would be even more difficult to measure the fatigue 
resistance and the recovery rate for an individual. Although a ex- 
perimental method has been proposed and realized in [33 1, this 
method is time-consuming. Furthermore, it would very expen- 
sive to obtain the fatigue resistance distribution of a given pop- 
ulation, since enormous measurements will be carried out. An 
much more efficient method has to be found before the valida- 
tion. Regarding the recovery rate, a large dispersion has been 
found in existing experimental models for determining work-rest 
schedule 11361 . How to characterize the recovery rate is a much 
more challenging task for us. Therefore, only a simulation task 
is provided for this case. All the related parameters were ob- 
tained from the literature or by theoretical analysis. However, it 
is believed that a guidance function could be provided. 

5 Conclusions and perspectives 

In this paper, an approach of integrating digital human sim- 
ulation into virtual reality application was introduced. The vir- 



TABLE 2. Endurance time evaluation of the drilling case. H: height; 
W: Weight; S: Joint moment strength 

Endurance time of the shoulder joint [second] 

Fatigue resistance 
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tual human is modeled from geometrical level to graphical level, 
to virtually represent a real human in the simulation system. It 
does not only provide a virtual representation, but also a tool 
for assessing different aspects of the task. In this study, inverse 
dynamics and the muscle fatigue and recovery model are inte- 
grated to predict the physical fatigue and endurance time at joint 
level. A case study was given for assessing physical fatigue of 
a drilling operation. Using virtual human simulation in VR sys- 
tem, it is promising to predict the fatigue of a given population 
under static cases in virtual environment. Furthermore, worker 
selection can also be accomplished in VR systems, if personal 
properties in physical fatigue could be well determined. 

Assessing manual handling operation in virtual environment 
is very useful for work design. The research mentioned in study 
was limited by the biomechanical model of the virtual human, 
because the fatigue resistance and the recovery rate for a given 
population have not been measured. We have made some effort 
and we will continue to do the research in order to find an effi- 
cient method to determine the fatigue properties of an individual. 
Once fatigue and recovery for an individual can be easily char- 
acterized by the measured fatigue resistance and recovery rate, 
a biomechanical database might be established for a given pop- 
ulation, or different population groups (e.g., industrial workers, 
nurses, etc.), and then physical assessment could be further per- 
formed for different physical tasks involving different population 
groups. 
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